CORUCTiaNtWM ^ FOURIER TRANSFORM 

CROSS REFERENCE TO RELATED APPLICATIONS 
,00011 This application is a continuation-in-part of U.S. application Ser. No. 

09/5.2,775 entitled "CORRECTION OF UNFOCUSED LENS EFFECTS VIA FRACTIONAL 
FOURIER TRANSFORM" filed February 25, 2000, which claims benefit of priority of U.S. 
provision, apptications Ser. Nos. 60/121,680 and 60/121,958, each filed on February 25, .999. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

|0 „021 This invention rc.ates .0 optica, signal processing, and more particularly .0 

in photographs, video, and other types of captured images. 

2. pjscussion of the Related Art 

,00031 A number of references are cited herein; these are provided in a numbered 

hst a, the end of the de.ai.ed description of ft. preferred embodiment, These references are cited 
al various .ocations throughout the specification using a reference number enc.osed in square 
brackets. 

,00041 The Fourier transforming properties of simple lenses and related optica. 

Cements is wd, known and heavi.y used in a branch of engineering known as Fourier optics 
„, 2] . Classical Fourier optics [.,2,3,4, utilize lenses or other means to obtain a two-dimensiona, 



Fourier transform 



of an optica! wavefiont, thus creating a Fourier p.ane a. a par.icu.ar spatial 
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lo ca,ion relative .0 an associated lens. This Fourier plane includes an amplitude distribution of 
an original two-dimensional optical image, which becomes the two-dimensional Fourier 
transform of itself. In the far simpler area of classical geometric optics [1,3], lenses and related 
objects are used to change the magnification of a two-dimensional image according to the 
geometric relationship of the classical lens-law. It has been shown that between the geometries 
required for classical Fourier optics and classical geometric optics, the action of a lens or related 
object acts on the amplitude distribution of images as the fractional power of the two- 
dimensional Fourier transform. The fractional power of the fractional Fourier transform is 
determined by the focal length characteristics of the lens, and the relative spatial separation 
between a lens, source image, and an observed image. 

[00051 The fractional Fourier transform has been independently discovered on 

various occasions over the years [5,7,8,9,10], and is related to several types of mathematical 
objects such as the Bargmam, transform [8] and the Hemti.e semigroup [13). As shown in [5], 
the most general form of optical properties of lenses and other related elements [1,2,3] can be 
transformed into a fractional Fourier transform representation. This property has apparently been 
rediscovered some years later and worked on steadily ever since (see for example [6]), expanding 
the number of optical elements and situattons covered. It is important to remark, however, that 
the lens modeling approach in the latter ongoing series of papers view the multiplicative phase 
tem, in the true form of the fractional Fourier transform as a problem or annoyance and usually 
omit it from consideration. 



2738-1 1 



-2- 



SUMMARY OF THE INVENTION 
,0,061 Correction of the effects of misfocusing in recorded or real-time image 

data may be accomplished nsing fractional Fourier transfom, operations reahzed optical.* 
computational or electronic*, hr some embodiments, me invention extends the capabilities 
of using a power of the fractional Fourier transform for correcting misfocused images, ,0 
situations where phase information associated with the original image misfocus is unavailable. 
For example, conventional photographic and electronic image capture, storage, and production 
technologies can only capture and process image amplitude information - the relative phase 
information created within the original optica, path is lost. As will he describe, herein, the 
ntissing phase tnfotmation can be recounted and used when correcting image misfocus. 

,00071 In accordance with some embodiments, algebraic group properties of the 

fractional Fourier transfom, are used to bacR-calculate los, original relative phase conditions that 
would have existed if a given specific corrective operation were to correct a misfocused image. 
Corrective iterations can men be made to converge on a corrected focus condition. Simplified 
numerica, calculations of phase reconstructions may be obtained by leveraging additional 
properties of the fractional Fourier transfer for employing pre-computed phase reconstructions. 

,0008) Some embodiments use the inherent fractional Fourier transform 

properties of .enses or related elements or environments, such as compound lenses or graded- 
index materials, to correct unfocused effects of various types of captured images. Use of the 
algebraic unitary group property of the fractional Fourier transfom, allows for a simple 
characterization of me exact inverse operation for the initial misfocus. 

,00091 One aspect of the present invention reconstructs retative phase information 

affUiatedwith the original misfocused optica, path mthe correction of misfocused images. 
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[0010] Another aspect of the present invention provides for the calculation of an 

associated reconstruction of relative phase information, which would be accurate if an associated 
trial fractional Fourier transform power were the one to correct the focus of the original 
misfocused image. 

,00111 Another aspect of the present invention provides for simplified calculation 

of the phase reconstruction information using algebraic group and antisymmetry properties of me 

fractional Fourier transform operator. 

[00121 Still yet another aspect of the present invention provides for simplified 

calculation of the phase reconstruction information using modified calculations of fractional 

powers of the Fourier transform. 

|0013] Another aspect of the present invention provides for simplified calculation 

of the phase reconstruction information by rearranging terms in the calculation of a fractional 

Fourier transform operator. 

[0014] Yet another aspect of the present invention provides for simplified 

calculation of the phase reconstruction information using a partition of terms in the calculation of 
a fractional Fourier transform operator. 

100151 Still yet another aspect of the present invention provides for relevant 

fractiona. Fourier transform operations to be accomplished directly or approximately by means of 
optical components, numerical computer, digital signal processing, or other signal processing 

methods or environments. 

[0016] Yet another aspect of the present invention provides approximation 

methods which leverage Hermite function expansions which can be advantageous in that the 
orthogonal Hermite functions diagonalize the Fourier transform and fractional Fourier transform, 
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yielding the following two-fold result. First, throughout the entire optical system, the amplitude 
and phase affairs of each Hermite function are completely independent of those of the other 
Hermite functions. Second, the Hermite function expansion of a desired transfer function will 
naturally have coefficients that eventually go to zero, meaning that to obtain an arbitrary degree 
of approximation in some situations, only a manageable number of Hermite functions need be 
considered. 

[0017] Another aspect of the present invention allows the power of the fractional 

Fourier transform to be determined by automatic methods. These automatic methods may 
include edge detection elements and provisions for partial or complete overriding by a human 
operator. 

[0018] Still yet another aspect of the present invention provides for the fraction 

Fourier transform power to be determined entirely by a human operator. 

[0019] Another aspect of the invention provides for pre-computed values of phase 

reconstructions corresponding to pre-computed powers of fractional Fourier transform to be 
composed to create phase reconstructions corresponding to other powers of the fractional 
Fourier transform. 

[0020] Another aspect of the present invention provides for at least one pre- 

computed power of the fractional Fourier transform to be used in computing or approximating 
higher powers of the fractional Fourier transform. 

[0021] Yet another aspect of the present invention provides for pre-computed 

values of phase reconstructions corresponding to powers of fractional Fourier transform, wherein 
the powers are related by roots of the number 2 or in other ways to leverage fractional expansion. 

[0022] Still yet another aspect of the present invention provides for composed 
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phase reconstructions that are realized .„ correspondence to binary representations of fraction, 
,00231 Yet another aspect of the present invention provides for combining 

numerical correction of video camera lens misfocns with video decompression algorithms to 

increase performance and rednce required misfocus-correction computation, 

,00241 The present invention enables the recovery of misfocused images obtained 

from photographs, video, movie, and other types of captured image, Because a high quality 
,ens or lens system operates on the amplitude distribution of the source image as a two- 
dmrensiona, fractional Fourier transform, the algebraic unitary group property of the fractional 
Fourier transform allows for the exact calculation of Ore inverse operation for initial lens 
nnsfocus. Additional mathematical properties of the fractional Fourier transform allow for 
different methods of approximation meaningful in the economic embodiments of the invention. 
The system and method provided herein enable economic and wide-ranging implementation for 
after-capture correction of image misfocu, for example. 

BRIEF DESCRIPTION OF THE DRAWINGS 
,00251 The above and other aspects, features and advantages of the present 

invention will become more apparent upon consideration of the following description of 
preferred embodiments taken in conjunction with the accompanying drawing figure, wherein: 
Figure 1 is a block diagram showing a general lens arrangement and associated image 
observation entity capable of classical geometric optics, classical Fourier optic, and fractional 

Fourier transform optics; 

Figure 2 is a block dia^am showing an exemplary approach for automated adjustment of 
fractional Fourier transform parameters for maximizing the sharp edge content of a corrected 
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image, in accordance with one embodiment of the present invention; 

Figure 3 is a block diagram showing a typical approach for adjusting the fractional 
Fourier transform parameters to maximize misfocus correction of an image, in accordance with 
one embodiment of the present invention; 

Figure 4 is a diagram showing a generalized optical environment for implementing image 

correction in accordance with the present invention; 

Figure 5 is a diagram showing focused and unfocused image planes in relationship to the 

optical environment depicted in figure 4; 

Figure 6 is a block diagram showing an exemplary image misfocus correction process 

that also provides phase corrections; 

Figure 7 is a diagram showing a more detailed view of the focused and unfocused image 

planes shown in Figure 5; 

Figure 8 is a diagram showing typical phase shifts involved in .he foensed and nnfocnsed 

image planes depicted in Figure 5; 

Figure 9 shows techniques for computing phase correction determined by the fractional 

Fourier transform applied to a misfocused image; and 

Figure 10 is a block diagram showing an exemplary image misfocus correction process 
that also provides for phase correction, in accordance with an alternative embodiment of the 
invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[00261 In the following description, reference is made to the accompanying 

drawing figures which form a part hereof, and which show by way of illustration specific 
embodiments of the invention. It is to be understood by those of ordinary skill in this 
technological field that other embodiments may be utilized, and structural, electrical, optical, as 
well as procedural changes may be made without departing from the scope of the present 
invention. 

[0027] As used herein, the term "image" refers to both still-images (such as 

photographs, video frames, video stills, movie frames, and the like) and moving nnages (such as 
motion video and movies). Many embodiments of the present invention are directed to 
processing recorded or real-time image data provided by an exogenous system, means, or 
method. Presented image data may be obtained from a suitable electronic display such as an 
LCD panel, CRT, LED array, films, slides, illuminated photographs, and the like. Alternatively 
or additionally, the presented image data may be the output of some exogenous system such as 
an optical computer or integrated optics device, to name a few. The presented image data will 
also be referred to herein as the image source. 

[0028] If desired, the system may output generated image data having some 

amount of misfocus correction. Generated image data may be presented to a person, sensor (such 
as a CCD image sensor, photo-transistor array, for example), or some exogenous system such as 
an optical computer, integrated optics device, and the like. The entity receiving generated image 
data will be referred to as an observer, image observation entity, or observation entity. 

[0029] Reference will first be made to Figure 3 which shows a general approach 

for adjusting the fractional Fourier transform parameters to maximize the correction of misfocus 
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in an image. Details regarding the use of a fractional Fourier transform (with adjusted parameters 
of exponential power and scale) to correct image misfocus will be later described with regard to 
Figures 1 and 2. 

[0030] Original visual scene 301 (or other image source) may be observed by 

optical system 302 (such as a camera and lens arrangement) to produce original image data 303a. 
In accordance with some embodiments, optical system 302 may be limited, misadjusted, or 
otherwise defective to the extent that it introduces a degree of misfocus into the image 
represented by the image data 303a. It is typically not possible or practical to correct this 
misfocus effect at optical system 302 to produce a better focused version of original image data 
303a. Misfocused original image data 303a may be stored over time or transported over 
distance. During such a process, the original image data may be transmitted, converted, 
compressed, decompressed, or otherwise degraded, resulting in an identical or perturbed version 
of original image data 303b. It is this perturbed version of the original image data that may be 
improved using the misfocus correction techniques disclosed herein. Original and perturbed 
image data 303a, 303b may be in the form of an electronic signal, data file, photography paper, 
or other image form. 

[0031] Original image data 303b may be manipulated numerically, optically, or 

by other means to perform a fractional Fourier transform operation 304 on the original image 
data to produce resulting (modified) image data 305. The parameters of exponential power and 
scale factors of the fractional Fourier transform operation 304 maybe adjusted 310 over some 
range of values, and each parameter setting within this range may result in a different version of 
resulting image data 305. As the level of misfocus correction progresses, the resulting image 
data 305 will appear more in focus. The improvement in focus will generally be obvious to an 



2738-1 1 



-9- 



attentive human visual observer, and will .ypically be signified by an inerease in image sharp- 
ness, par.ien.arly a, any edges .ha. appear in the image. Thns a hnman opem«or, a maehine 
eon.ro. system, or a combination of eaeh ean eompare a sequence of resnUing images erea.ed by 
previously se.ee«ed parameter settings 310, and try a new parameter setting for a ye. another 

potential improvement. 

10 OJ2] For a human operator, this typically would be a matter of adjusting a 

eon.ro. and comparing images side by side (facilitated by non-human memory) or, as in me ease 
of a microscope or telescope, by comparison facilitated purely with human memory. For a 
machine, a systematic iterative or other feedback control scheme would .ypically be used. 
In Figure 3, eaeh of these image adjustments is generalized by .he s,eps and e.emenls sngges.ed 
by in.entonnec.ed dements 306-309, ahhough other systems or memods accomplishing the same 
goal with different interna, sttucttrre (for example, an analog electronic circuit, optical materials, 
or chemical process) are provided for and an.icipa.ed by the present invention. For the 
iHustrauve general case of Figure 3, resulting image data 305 for selected parameter settings 310 
maybe stored in human, machine, or photographic memory 306, along with .he associated 
parameter settings, and compared 307 for .he quality of image focus. Based on these 
comparisons, subsequent high level actions 308 may be chosen. 

(00331 High level actions 308 typically require translation into new parameter 

values and then realization, which may be provided by parameter calculation and control 309. 
This process may continue for some interval of time, some number of resulting images 305, or 
some chosen or predetermined maximum level of improvement. One or more "best choice- 
resulting image data set or sels 305 may .hen be identified as me result of .he action and 
processes depicted in this figure. 
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,0034] With this high level description having been established, attention is now 

directed to details of the properties and use of a fractional Fourier transform (with adjusted 
parameters of exponential power and scale) to correct misfocus in an image and maximize 
correction of misfocus. This aspect of the present invention will be described with regard to 
Figure 1. 

(0035] Figure 1 is a block diagram showing image source 101, lens 102, and 

image observation entity 103. The term "lens" is used herein for convenience, but it is to be 
understood that the image misfocus correction techniques disclosed herein apply equally to lens 
systems and other similar optical environment, The image observation entity may be configured 
with classical geometric optics, classical Fourier optics, or fractional Founer transform optic, 
The particular class of optics (geometric, Fourier, or fractional Fourier) .mplemented in a certain 
application may be determined using any of the following: 

• separation distances 111 and 112; 

• the "focal length" parameter "f of lens 102; 

. the type of image source (lit object, projection screen, etc.) in as far as whether a 

plane or spherical wave is emitted. 

[0036] As is well known, in situations where the source image is a lit object and 

where distance 111, which shall be called "a," and distance 112, which shall be called "b," fall 



I = i + 1 (1) 
fab 



into 



the lens-law relationship, may be determined by the focal length/: 
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which gives .he geometric optics case, b this case, observed image 103 is a vertically and 
horizontally inverted version of rhe origina. image from source 101, scaled in size by a 
magnification factor m given by: 

b (2) 
a 

[00371 As previously noted, the Fourier transforming properties of simple lenses 

and related optical elements is also well known in the field of Founer optics [2,3]. Classical 
Fourier optics [2,3,4,5] involve the use of a lens, for example, to take a first two-dimensional 
Fourier transform of an optical wavefront, thus creating a Fourier plane at a particular spatial 
location such that the amplitude distribution of an original two-dimensional optical image 
becomes the two-dimensional Fourier transform of itself. In the arrangement depicted in Figure 

1, with a lit object serving as source image 101, the Fourier optics case may be obtained when 

a = b=f. 

[0 038] As described in [5], for cases where a, b, and f do not satisfy the lens law 

of the Fourier optics condition above, the amplitude distribution of source image 101, as 
observed at observation entity 1 03, experienees the action of a non-integer power of the Founer 
transform operator. As described in [5], this power, which shall be called a, varies between 0 
and 2 and is determined by an Arc-Cosine function dependent on the lens focal length and the 
distances between the lens, image source, and image observer; specifically: 



2 

a = - arccos 
% 



[sgn^^SnQL] (3, 
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fc cases where </-„) and <W share the same sign. There are Cher cases which can he so.ved 
from ,he more primitive ^nations in [5, (a. .he botiom of pages ThE4-3 and ThE4-.). Note 
simp ,e suhs.Un.ions show tha, .he .ens .aw reiationship among a, ». and/indeed give a power of 
2, and .ha. .he Fonrier optics condi.ion of a - b =/give .he power of 1, as required. 

(00391 The fractional Fonrier transform properties oflenses lypically cause 

complex but predictable phase and scale variations. These variations may be expressed in terms 
of Henuite functions, as presenter, shortly, bu, i. is unders.ood tha, other repressions of .he 
effects, such as closed-form Integra, repressions g .ven in [5], are a.so possible and useful 

,00401 Several methods can be used <o construe, .he fractiona. Founer transform, 

but ,„ begin it is Ulustrative to use the orthogonal Hermi.e functions, which as eigenlhnctions 
diagonalize me Fourier transform [17], Consider me Hermue taction [.6] expansion [17, and 
more recent.y, 18] of.be tivo dimensional image amp.itnde dis.ribu.ion function. In one dimen- 
sion, a bounded (i.e., non-infinite) function k(x, can be represent as an mfinire sum of Hemti.e 
functions {h n (x)} as: 



CO 

(4) 



« = 0 



l00 41] Since the function is bounded, the coefficients {a n} eventually become 

smaller and converge to zero. An image may be treated as a two dhnensional entity (for 
example, atwo—^ 

plate; in either case, the function may be represented in a two-dimensional expansion such as: 
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00 oo 



m 



= 0«=0 



For simplicity, .he one dimensional case may be considered. The Fourier transform action on 
Hemrite expansion of the function Hf) with series coefficients {a„}is given by [16]: 



00 



n = 0 



Because of the diagonal eigenfunction structure, fractional powers of the Fourier transform 
operator may be obtained by taking the fractional power of each eigenfunction coefficient. The 
eigenfunction coefficients here are (-if. Complex branching artifact ambiguities that arise from 
taking the roots of complex numbers can be avoided by writing (- 0 as: 

-in/2 ( ? ) 

e 

Thus for a given power a, the fractional Fourier transform of the Hermite expansion of the 
function k(x) with series coefficients {a n }can be given by [5]: 



(8) 
n = 0 
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Note when a = 1 , the result is the traditional Fourier transform above, and when a = 2, the result 
may be expressed as: 



2 [*<*)]= i ""W*) = i H)V«(*> = i *«v-*> = *<-*> (9) 

«=o »=o n=0 



due to the odd and even symmetry, respectively, of the odd and even Hermite functions. Tins is 
the case for the horizontally and vertically inverted image associated with the lens law of 
geometric optics, although here the scale factors determining the magnification factor have been 
normalized out. 

[0042] More generally, as the power a varies (via the Arccosine relationship 

depending on the separation distance), the phase angle of the n* coefficient of the Hermite 
expansion varies according to the relationship shown above and the scale factor may vary as well 
[5]. For images, all of the above occurs in the same manner but in two dimensions [5]. 

[0043] Through use of the Mehler kernel [16], the above expansion may be 

represented in closed form as [5]: 

Ali sin (na /2) 

Note in [5] that the factor of i multiplying the sin function under the radical has been 
erroneously omitted. Clearly, both the Hermite and integral representations are periodic in a 
with period four. Further, it can be seen from either representation that: 
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F 2±e [k(x)] = p2 F ±e [k(x)] = f ±e p 2 [ k ( x) ] = F ±£ [k(-x)] (1 1) 

which illustrates an aspect of the invention as the effect 8 will be the degree ofmisfocus 
introduced by the misfocused lens, while the Fourier transform raised to the second power 
represents the lens-law optics case. In particular, the group property makes it possible to 
calculate the inverse operation to the effect induced on a record image by a misfocused lens in 
terms of explicit mathematical operations that can be realized either computationally, by means 
of an optical system, or both. Specifically, because the group has period 4, it follows that 
F' 2 =F 2 ; thus: 

i^'mx)]) 1 = F- 2 F~[k(x)}= ^F^mx)) = ft [*(*)]= [*(-*)] ( 12 ) 



Thus, one aspect of the invention provides image misfocus correction, where the misfocused 
image had been created by a quality though misfocused lens or lens-system. This misfocus can 
be corrected by applying a fractional Fourier transform operation; and more specifically, if the 
lens is misfocused by an amount corresponding to the fractional Fourier transform of power e, 
the misfocus may be corrected by applying a fractional Fourier transform operation of power - e. 

[0044] It is understood that in some types of situations, spatial scale factors of the 

image may need to be adjusted in conjunction with the fractional Fourier transform power. For 
small variations of the fractional Fourier transform power associated with a slight misfocus, this 
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is unlikely to be necessary. However, should spatial scaling need to be made, various optical and 
signal processing methods well known to those skilled in the art can be incorporated. In the case 
of pixilated images (images generated by digital cameras, for example) or lined-images 
(generated by video-based systems, for example), numerical signal processing operations may 
require standard resampling (interpolation and/or decimation) as is well known to those familiar 

with standard signal processing techniques. 

[0045] It is likely that the value of power e is unknown a priori. In this particular 

circumstance, the power of the correcting fractional Fourier transform operation may be varied 
until the resulting image is optimally sharpened. This variation could be done by human 
interaction, as with conventional human interaction of lens focus adjustments on a camera or 

microscope, for example. 

10046] If desired, this variation could be automated using, for example, some sort 

of detector in an overall negative feedback situation. In particular, it is noted that a function with 
sharp edges are obtained only when its contributing, smoothly-shaped basis functions have very 
particular phase adjustments, and perturbations of these phase relationships rapidly smooth and 
disperse the sharpness of the edges. Most natural images contain some non-zero content of sharp 
edges, and further it would be quite unlikely that a naturally occurring, smooth gradient would 
tighten into a sharp edge under the action of the fractional Fourier transform because of the 
extraordinary basis phase relationships required. This suggests that a spatial high-pass filter, 
differentiator, or other edge detector could be used as part of the sensor makeup. In particular, an 
automatically adjusting system may be configured to adjust the fractional Fourier transform 
power to maximize the sharp edge content of the resulting correcting image. If desired, such a 
system may also be configured with human override capabilities to facilitate pathological image 
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situations, for example. 

[0047] Figure 2 shows an automated approach for adjusting the fractional Fourier 

transform parameters of exponential power and scale factor to maximize the sharp edge content 
of the resulting correcting image. In this figure, original image data 201 is presented to an 
adjustable fractional Fourier transform element 202, which may be realized physically via optical 
processes or numerically (using an image processing or computation system, for example). The 
power and scale factors of the fractional Fourier transform may be set and adjusted 203 as 
necessary under the control of a step direction and size control element 204. 

[0048] Typically, this element would initially set the power to the ideal value of 

zero (making the resulting image data 205 equivalent to the original image data 201) or two 
(making the resulting image data 205 equivalent to an inverted image of original image data 
201), and then deviate slightly in either direction from this initial value. The resulting image 
data 205 may be presented to edge detector 206 which identifies edges, via differentiation or 
other means, whose sharpness passes a specified fixed or adaptive threshold. The identified edge 
information may be passed to an edge percentage tally element 207, which transforms this 
information into a scalar-valued measure of the relative degree of the amount of edges, using this 

as a measure of image sharpness. 

[0049] The scalar measure value for each fractional Fourier transform power may 

be stored in memory 208, and presented to step direction and size control element 204. The step 
direction and size control element compares this value with the information stored in memory 
208 and adjusts the choice of the next value of fractional Fourier transform power accordingly. 
In some implementations, the step direction and size control element may also control edge 
detection parameters, such as the sharpness threshold of edge detector element 207. When the 
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optimal adjustment is determined, image data 205 associated with the optimal fractional Fourier 
transform power is designated as the corrected image. 

[0050] It is understood that the above system amounts to a negative-feedback 

control or adaptive control system with a fixed or adaptive observer. As such, it is understood 
that alternate means of realizing this automated adjustment can be applied by those skilled in the 
art. It is also clear to one skilled in the art that various means of interactive human intervention 
may be introduced into this automatic system to handle problem cases or as a full replacement for 

the automated system. 

[0051] In general, the corrective fractional Fourier transform operation can be 

accomplished by any one or combination of optical, numerical computer, or digital signal 
processing methods as known to those familiar with the art, recognizing yet other methods may 
also be possible. Optical methods may give effectively exact implementations of the fractional 
Fourier transforms, or in some instances, approximate implementations of the transforms. For a 
pixilated image, numerical or other signal processing methods may give exact implementations 
through use of the discrete version of the fractional Fourier transform [10]. 

[0052] Additional computation methods that are possible include one or more of: 

. dropping the leading scalar complex-valued phase term (which typically has little 

or no effect on the image); 
. decomposing the fractional Fourier transform as a pre-multiplication by a "phase 
chirp" e icz \ taking a conventional Fourier transform with appropriately scaled 
variables, and multiplying the result by another "phase chirp;" and 
• changing coordinate systems to Wigner form: 
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If desired, any of these just-described computation methods can be used with the approximating 

methods described below. 

[0053] Other embodiments provide approximation methods for realizing the 

corrective fractional Fourier transform operation. For a non-pixilated image, numerical or other 
gnal processing methods can give approximations through: 

. finite-order discrete approximations of the integral representation; 

. finite-term discrete approximations by means of the Hermite expansion 

representation; and 
. the discrete version of the fractional Fourier transform [10]. 
[0054] Classical approximation methods [11,12] may be used in the latter two 

cases to accommodate particular engineering, quality, or cost considerations. 

[0055] In the case of Hermite expansions, the number of included terms may be 

determined by analyzing the Hermite expansion of the image data, should this be tractable. In 
general, there will be some value in situations where the Hermite function expansion of the 
image looses amplitude as the order of the Hermite functions increases. Hermite function orders 
with zero or near-zero amplitudes may be neglected entirely from the fractional Fourier 
mputation due to the eigenfunction role of the Hermite functions in the fractional Fourier 



cor 



transform operator. 

[0056] One method for realizing finite-order discrete approximations of the 
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integral representation would be to employ a localized perturbation or Taylor series expansion of 
the integral representation. In principal, mis approach typtcaUy requires some mathematical care 
in order for .he operator ,0 act as a reflection operator (i.e., inversion of each horizontal direction 
and vertical direction as with the lens law) since the kernel behaves as a generalized function 
(delta function), and hence the integraf representation of the fractional Fourier transform operator 

resembles a singular integral. 

[00571 In a compound lens or other composite optical system, the reflection 

operator may be replaced with the identity operator, which also involves virtually identical delta 
functions and singular integrals as is known to those familiar in the art. However, this situation 
is fairly easy to handle as a first or second-order Taylor series expansion. The required first, 
second, and any higher-order derivatives of the fractional Fourier transform integral operator are 
readily and accurately obtained symbolically using available mathematical software programs, 
such as Mathematica or MathLab, with symbolic differential calculus capabilities. In most cases, 
the zero-order term in the expansion will be the simple reflection or identity operator. The 
resulting expansion may then be numerically approximated using conventional methods. 

[00581 Another method for realizing finite-order discrete approximations of the 

integral representation would be to employ the infinitesimal generator of the fractional Fourier 
transform, that is, the derivative of the fractional Fourier transform with respect to the power of 
the transform. This is readily computed by differentiating the Hermite function expansion of the 
fractional Fourier transform, and use of the derivative rule for Hermite functions. Depending on 
the representation used [5,14,15], the infinitesimal generator maybe formed as a linear 
combination of the Hamiltonian operator H and the identity operator I; for the form of the 
integral representation used earlier, this would be: 
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L*(H+I) (14) 
4 



where and the identity operator I simply reproduces the original Amotion, and 



X 



The role of the infinitesimal generator, which can be denoted as A, is to represent an operator 
group in exponential form, a particular example is: 



a aA (16) 
F = e 



For small values of A, one can then approximate e aA as I + (oA) , so using the fact [12] from 
before (repeated here): 



i.1 _>o-i 1 



-zz- 



(^ ±8 [*(*)]) -1 = ^Vm- ^\k(x)] = F* £ F[k{x)]= F*[k(-x)] M 



one can then approximate as 



F £ = I + (e^I + eij(^-, 2 + l) 08) 



These operations can be readily applied to images using conventional image processing methods. 

[0059] For non-pixilated images, the original source image can be approximated 

by two-dimensional sampling, and the resulting pixilated image can then be subjected to the 
discrete version of the fractional Fourier transform [10]. 

[0060] In cases where the discrete version of the fractional Fourier transform [10] 

implemented, the transform may be approximated. The discrete representation can, for 
ample, be a three-dimensional matrix (tensor) operator. Alternatively, pairs of standard two- 
dimensional matrices, one for each dimension of the image, can be used. As with the continuous 
case, various types of analogous series approximations, such as those above, can be used. 

[0061] Finally, it is noted that because of the commutative group property of the 

fractional Fourier transform, the matrix/tensor representations, or in some realizations even the 



IS 
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integrals cited above may be approximated by pre-computing one or more fixed step sizes and 
applying these respectively, iteratively, or in mixed succession to the image data. 

[0062] One exemplary embodiment utilizing a pre-computation technique may be 

where the fractional Fourier transform represents pre-computed, positive and negative values of a 
small power, for example 0.01. Negative power deviations of increasing power can be had by 
iteratively applying the pre-computed -0.01 power fractional Fourier transform; for example, the 
power -0.05 would be realized by applying the pre-computed -0.01 power fractional Fourier 
transform five times. In some cases of adaptive system realizations, it may be advantageous to 
discard some of the resulting image data from previous power calculations. This may be 

accomplished by backing up to a slightly less negative power by applying the +0.01 power 

fractional Fourier transform to a last stored, resulting image. 

[0063] As a second example of this pre-computation method, pre-computed 

fractional Fourier transform powers obtained from values of the series 2 m and T m may be 

stored or otherwise made available, for example: 



±1 /1024 ±1/512 ±1 /256 ±1/128 ±1'<>4 , ( 19 ) 

{F ,F ,F ,F ,t ,.-.} 
Then, for example, the power 1 1/1024 can be realized by operating on the image data with 



1/1024 1 /256 1 /128 ( 20 ) 
F F F 



where the pre-computed operators used are determined by the binary-decomposition of the power 
with respect to the smallest power value (here, the smallest value is 1/1024 and the binary 
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decomposition of 1 1/1024 is 1/1024 + 1/256 +1/128, following from the fact that 1 1 = 8 + 2 + 1). 

Such an approach allows, for example, N steps of resolution to be obtained from a maximum of 

lo&N compositions of log 2 N pre-computed values. 

[0064] It is noted that any of the aforementioned systems and methods may be 

adapted for use on portions of an image rather than the entire image. This permits corrections of 
localized optical aberrations. In complicated optical aberration situations, more than one portion 
of an image may be processed in this manner, with differing corrective operations made for each 
portion of the image. 

[0065] Finally, it is noted that the systems and methods described herein may also 

be applied to conventional lens-based optical image processing systems, to systems with other 
types of elements obeying fractional Fourier optical models, as well as to widely ranging envi- 
ronments such as integrated optics, optical computing systems, particle beam systems, electron 
microscopes, radiation accelerators, and astronomical observation methods, among others. 

[0066] Commercial products and services application are widespread. For 

example, the present invention may be incorporated into film processing machines, desktop 
photo editing software, photo editing web sites, VCRs, camcorders, desktop video editing 
systems, video surveillance systems, video conferencing systems, as well as in other types of 
products and service facilities. Four exemplary consumer-based applications are now considered. 
[0067] 1 • One particular consumer-based application is in the correction of 

camera misfocus in chemical or digital photography. Here the invention may be used to 
process the image optically or digitally, or some combination thereof, to correct the misfocus 
effect and create an improved image which is then used to produce a new chemical 
photograph or digital image data file. In this application area, the invention can be 
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incorporated into film processing machines, desktop photo editing software, photo editing 

web sites, and the like. 

[00681 2. Another possible consumer-based application is the correction of video 

camcorder misfocus. Camcorder misfocus typically results from user error, design defects 
such as a poorly designed zoom lens, or because an autofocus function is autoranging on the 
wrong part of the scene being recorded. Non-varying misfocus can be corrected for each 
image with the same correction parameters. In the case of zoom lens misfocus, each frame or 
portion of the video may require differing correction parameters. In this application area, the 
invention can be incorporated into VCRs, camcorders, video editing systems, video 
processing machines, desktop video editing software, and video editing web sites, among 
others. 

(00691 3. Another commercial application involves the correction of image 

misfocus experienced in remote video cameras utilizing digital signal processing. Particular 
examples include video conference cameras or security cameras. In these scenarios, the 
video camera focus cannot be adequately or accessibly adjusted, and the video signal may in 
fact be compressed. 

[0070] 4. Video compression may involve motion compensation operations that 

were performed on the unfocused video image. Typical applications utilizing video 
compression include, for example, video conferencing, video mail, and web-based video-on- 
demand, to name a few. In these particular types of applications, the invention maybe 
employed at the video receiver, or at some pre-processing stage prior to delivenng the signal 
to the video receiver. If the video compression introduces a limited number of artifacts, 
misfocus correction is accomplished as presented herein. However, if the video compression 
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introduces a higher number of artifacts, the signal processing involved with the invention 
may greatly benefit from working closely with the video decompression signal processing. 
One particular implementation is where misfocus corrections are firs, applied to a full video 
frame image. Then, for some interval of time, misfoens correction is only applied to the 
changing regions of the video image. A specific example may be where .arge portions of a 
misfocuscd background can be corrected once, and then reused in those same regions in 

subsequent video frames. 

[0071] 5. The misfocus correction techniques described herein are directly 

applicable to electron microscopy systems and applications. For example, electron 
microscope optics employ the wave properties of electrons to create a coherent optics 
environment that obeys the Fourier optics structures as coherent light (see, for example, John 
C.H. Spence, High-Resolution Electron Microscopy, third edition, 2003, Chapters 2-4, pp. 
15-88). Electron beams found in electron microscopes have the same geometric, optical 
phys.es characteristics generally found in coherent light, and the same mathematical 
quadratic phase structure as indicated in Levi [1] Section 19.2 for coherent light, which is the 
basis of the fractional Fourier transform in optical systems (see, for example, John C.H. 
Spence High-Resolution Electron Microscopy, third edition, 2003, Chapter 3, formula 3.9, 

pg. 55). 

Misfocused Optical Path Phase Reconstruction 
[0072] Most photographic and electronic image capture, storage, and production 

technologies are only designed to operate with image amplitude information, regardless as to 
whether the phase of the light is phase coherent (as is the case with lasers) or phase noncoherent 
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(as generally found in most light sources). In sharply focused images involving noncoherent 

light formed by classical geometric optics, this lack of phase information is essentially of no 

consequence in many applications. 

[00731 In representing the spatial distribution of light, the phase coefficient of the 

basis functions can be important; as an example, Figure 3.6, p. 62 of Digital Image Processing 
- Concepts, Algorithms, and Scientific Applications, by Bernd Jahne, Springer-Verlag, New 
York, 1991 [20] shows the effect of loss and modification of basis function phase information 
and the resulting distortion in the image. Note in this case the phase information of the light in 
the original or reproduced image differs from the phase information applied to basis functions 

used for representing the image. 

[0074] In using fractional powers of the Fourier transform to represent optical 

operations, the fractional Fourier transform reorganizes the spatial distribution of an image and 
the phase information as well. Here the basis functions serve to represent the spatial distribution 
of light in a physical system and the phase of the complex coefficients multiplying each of the 
basis functions mathematically result from the fractional Fourier transform operation. In the 
calculation that leads to the fractional Fourier transform representation of a lens, complex-valued 
coefficients arise from the explicit accounting for phase shifts of light that occurs as it travels 
through the optical lens (see Goodman [2], pages 77-96, and Levi [1], pages 779-784). 

[0075] Thus, when correcting misfocused images using fractional powers of the 

Fourier transform, the need may arise for the reconstruction of relative phase information that 
was lost by photographic and electronic image capture, storage, and production technologies that 
only capture and process image amplitude information. 

[0076] In general, reconstruction of lost phase information has not previously 
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been accomplished with much success, but some embodiments of the invention leverage specific 
properties of both the fractional Fourier transform and an ideal correction condition. More 
specifically, what is provided - for each given value of the focus correction parameter - is the 
calculation of an associated reconstruction of the relative phase information. Typically, the 
associated reconstruction will be inaccurate unless the given value of the focus correction 
parameter is one that will indeed correct the focus of the original misfocused image. 

[00771 This particular aspect of the invention provides for the calculation of an 

associated reconstruction of relative phase information by using the algebraic group property of 
the fractional Fourier transform to back calculate the lost relative phase conditions that would 
have existed, if that given specific focus correction setting resulted in a correctly focused image. 
For convergence of human or machine iterations towards an optimal or near optimal focus 
correction, the system may also leverage the continuity of variation of the phase reconstruction as 
the focus correction parameter is varied in the iterations. 

[0078] To facilitate an understanding of the phase reconstruction aspect of the 

invention, it is helpful to briefly summarize the some of the image misfocus correction aspects of 
the invention. This summary will be made with reference to the various optical set-ups depicted 
in Figures 4-8, and is intended to provide observational details and examples of where and how 
the relative phase reconstruction may be calculated (Figure 9) and applied (Figure 10). 

Misfocus Correction 
[00791 Figure 4 shows a general optical environment involving sources of 

radiating light 400, a resulting original light organization (propagation direction, amplitude, and 
phase) 401 and its constituent photons. Optical element 402 is shown performing an image- 
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forming optical operation, causing a modified light organization (propagation direction, 
amplitude, and phase) 403 and its constituent photons, ultimate.y resulting in observed image 
404. This figure shows that for each light organization 401, 403 of light and photons, the 
propagation direction, amplitude, and phase may be determined by a variety of different factors. 
For example, for a given propagation media, propagation direction, amplitude, and phase may be 
determined by such things as the separation distance between point light source 400 and optical 
element 402, the pixel location in a transverse plane parallel to the direction of propagation, and 
light frequency/wavelength, among others. 

10080) Figure 5 is an optical environment similar to that depicted in Figure 4, but 

the Figure 5 environment includes only a single point light source 500. In this Figure, single 
point light source 500 includes exemplary propagation rays 501a, 501b, 501c that are presented 
to optical element 502. The optical element is shown imposing an optical operation on these 
rays, causing them to change direction 503a, 503b, 503c. Each of the rays 503a, 503b, 503c are 
shown spatially reconvening a. a single point in the plane of image formation 504, which is a 
focused image plane. 

,00811 Figure 5 also shows direction* modified rays 503a, 503b, 503c spatially 

diverging a. short unfocused image plane 505 and long unfocused image plane 506, which are 
each transverse to the direction of propagation that is associated with images which are no. in 
sharp focus, which will be referred to as nonfocused image planes. Further description of the 
optical environment shown in Figure 5 w,U be presented to expand on phase cotrection, and such 
description will be later discussed with regard to Figures 9-10. 

|00821 Reference is now made to Figures 6-8, which disclose techniques for 

mathematica. focus correction and provides a basis for understanding the phase correction aspect 
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of the present invention. For clarity, the term "lens" will be used to refer to optical element 502, 
but the discussion applies equally to other types of optical elements such as a system of lenses, 

graded-index material, and the like. 

[0083] Figure 6 provides an example of image information flow in accordance 

with some embodiments of the present invention. As depicted in block 600, an original 
misfocused image is adapted or converted as may be necessary into a digital file representation of 
light amplitude values 601. Examples of original misfocused images include natural or 
photographic images. Digital file 601 may include compressed or uncompressed image format, 

l00 84] For a monochrome image, the light amplitude values are typically 

represented as scalar quantities, while color images typically involve vector quantities such as 
RBG values, YUV values, and the like. In some instances, the digital file may have been 
subjected to file processes such as compression, decompression, color model transformations, or 
other data modification processes to be rendered in the form of an array of light amplitude values 
602, Monochrome images typically only include a single array of scalar values 602a. In 
contrast, color images may require one, two, or more additional arrays, such as arrays 602b and 
602c. A CMYB color model is a particular example of a multiple array, color image. 

[0085] The array, or in some instances, arrays of light amplitude values 602 may 

then be operated on by a fractional power of the Fourier transform operation 603. This operation 
mathematically compensates for lens misfocus causing the focus problems in the original 
misfocused image 600. A result of this operation produces corrected array 604, and in case of a 
color model, exemplary subarrays 604a, 604b, 604c result from the separate application of the 
fractional power of the Fourier transform operation 603 to exemplary subarrays 602a, 602b, 
602c. If desired, each of the corrected subarrays 604a, 604b, 604c maybe converted into a 
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digital file repression of .he corrected image 605; this digital file could be the same format, 
similar format, or an entirely different forma, from .ha. of uncorrected, original digital file 
representation 601. 

(0086) Figure 7 shows an optical environment having nonfocused planes. This 

figure shows .ha. .he power of the fractional Fourier .ransfotm operator increases as ,he 
separation distance between optica, lens operation 502 and image planes 504, 505 increases, up 
,„ a dis.ance ma«chi„g tot of .he lens law. In accordance with some aspects of .he invention, 
and as explained in Ludwig [5], Goodman [2], pages 77-96, and Levi [1], pages 779-784, an 
exactly focused image corresponds to a fractional Fourier transform power of exactly two. 
Furthermore, as previously described, misfocused image plane 505 lies short of the focuseo 
image plane 504, and corresponds to a fractional Fourier transform operation with a power 
sHghtly less ton two. The deviation in .he power of to fractional Fourier .ransform operation 
corresponding to short misfocus image plane 505 will be denoted (-r, ), where to subscript »S" 
denotes "short." Since an exactly focuseo image at focused image plane 504 corresponds to a 
fractional Fourier transferor power of exactly two, this short misfocus may be corrected by 
application of the fractional Fourier transform raised ,0 to power ( «, ), as indicated in block 



701. 



[00871 By mathematical extension, as described in [5], a long misfocused image 

p,a„e 506 to, lies a, a distance further away from optical element 502 ton does .he focused 
image plane 504 would correspond to a fractional Fourier transform operation with a power 
slightly greater ton two. The deviation in the power of to fractional Fourier fransform 
operation corresponding to .ong misfocus image plane 506 will be denoted (4* ), where to 
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subscript "L" denotes "long." This long misfocus may be corrected by application of the 
fractional Fourier transform raised to the power (-8 L ), as indicated in block 702. 

Relative Phase Information in the Misfocused Optical Path 
[0088] In terms of geometric optics, misfocus present in short misfocused image 

plane 505 and long misfocused image plane 506 generally correspond to non-convergence of rays 
traced from point light source 500, through optical element 502, resulting in misfocused images 
planes 505 and 506. For example, figures 5 and 7 show exemplary rays 501a, 501b, 501c 
diverging from point light source 500, passing through optical element 502, and emerging as 
redirected rays 503a, 503b, 503c. The redirected rays are shown converging at a common point 
in focused image plane 504. However, it is important to note that these redirected rays converge 
at discreetly different points on misfocused image planes 505 and 506. 

[0089] Figure 8 is a more detailed view of image planes 504, 505 and 506. In this 

figure, rays 503a, 503b, 503c are shown relative to focused image plane 504, and misfocused 
image planes 505 and 506. This figure further shows the path length differences that lead to 
phase shifts of the focused and unfocused planes result from varying angles of incidence, denoted 
by 0, and 0 2 . The distances of rays 503a, 503b, 503c from optical element 502 are given by the 
following table: 
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Table 1 


Ray 


Distance to incidence 
with snort misiocubcu 
plane 505 


Distance to incidence D 
with focus olane 504 ! 

1 


Distance to incidence 
with long misfocused 
plane 506 


503a 


8, s 




8n L 


503b 


5o S 


8o F 


8o L 


503c 


8 2 s 


82- 


8 2 L 



Simple geometry yields the following inequality relationships: 



do < 8o F < 8o L ^ ^ 

6, S <5, F <8, L (22) 
8 2 S <8/<8 2 L (23) 



wavelength X, the phase shift V created by a distance-of-travel variation 8 is given by 

(24) 



For a given 
the following: 

y = 2n8 / X 

so the variation in separation distance between the focus mage plane 504 and the misfocus 
image planes 505, 506 is seen to introduce phase shifts along each ray. 

[0090] Further, for n/2 > 6, > 0 2 > 0, as is the case shown in Figure 8, simple 

trigonometry gives: 

5 0 F = 8, F sm9, (25) 

(26) 
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6 0 F = 5 2 F sinQ 2 

\>sinQi>sinQ 2 >0 ^ 
which in turn yields the inequality relationships: 

S 0 <5 2 <oi 

sF .» -f (29) 

(30) 

5 0 L <5 2 L <5, L 

Again, for a given wavelength X, the phase shift V created by a distance-of-travel variation 5 is 
given by the following: 

(31) 

v|/ = 2n8 / X 

so the variation in separation distance between focused image plane 504 and the misfocused 
image planes 505, 506 is seen to introduce non-uniform phase shifts along each ray. Thus the 
misfocus of the original optical path involved in creating the original image (for example, 600 in 
Figure 6) introduces a non-uniform phase shift across the rays of various incident angles, and this 
phase shift varies with the distance of separation between the positions of misfocused image 
planes 505, 506, and the focused image plane 504. 

[00911 Referring again to Figures 6 and 7, an example of how a misfocused image 

600 may be corrected will now be described. A misfocused image requiring correction will 
originate either from short misfocused plane 505 or long misfocused plane 506. In situations 
where misfocused image 600 originates from short misfocused plane 505, misfocus correction 
may be obtained by applying a fractional Fourier transform operation raised to the power ( + 8 S ), 
as indicated in block 701. On the other hand, in situations where misfocused image 600 
originates from long misfocused plane 506, misfocus correction may be obtained by applying a 
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fractional Fourier transform operation raised to the power (-6 L ), as indicated in block 702. 

[0092] hi general, the fractional Fourier transform operation creates results that 

are complex-valued. In the case of the discrete fractional Fourier transform operation, as used 
herein, this operation may be implemented as, or is equivalent to, a generalized, complex-valued 
array multiplication on the array image of light amplitudes (e.g., *). In the signal domain, 
complex-valued multiplication of a light amplitude array element, Oy, by a complex-valued 
operator element f results in an amplitude scaling corresponding to the polar or phasor 
amplitude off and a phase shift corresponding to the polar or phasor phase off 

[0093] Figure 9 shows a series of formulas that may be used in accordance with 

the present invention. As indicated in block 901, the fractional Fourier transform operation array 
(FrFT) is symbolically represented as the product of an amplitude information array component 
and a phase information array component. The remaining portions of Figure 9 illustrate one 
technique for computing phase correction in conjunction with the correction of image misfocus. 
For example, block 902 shows one approach for correcting image misfocus, but this particular 
technique does not provide for phase correction. Image correction may proceed by first noting 
that the nusfocused optics corresponds to a fractional Fourier transform of power 2-8 for some 
unknown value of s; here 8 may be positive (short-misfocus) or negative (long-misfocus). Next, 
the fractional Fourier transform may be mathematically applied with various, systematically 
selected trial powers until a suitable trial power is found. A particular example may be where the 
trial power is effectively equal to the unknown value of 8. The resulting mathematically 
corrected image appears in focus and a corrected image is thus produced. 

[0094] Referring still to Figure 9, block 903 depicts the misfocus correction 
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technique of block 901, as applied to the approach shown in block 902. This particular technique 
accounts for the amplitude and phase components of the optical and mathematical fractional 
Fourier transform operations. In particular, there is an amplitude and phase for the misfocused 
optics, which led to the original misfocused image 600. 

[0095] As previously noted, conventional photographic and electronic image 

capture, storage, and production technologies typically only process or use image amplitude 
information, and were phase information is not required or desired. In these types of systems, the 
relative phase information created within the original misfocused optical path is lost since 
amplitude information is the only image information that is conveyed. This particular scenario is 
depicted in block 904, which shows original misfocused image 600 undergoing misfocus 
correction, even thought its relative phase information is not available. In all applicable cases 
relevant to the present invention (for example, 0 < 8 < 2), the phase information is not uniformly 
zero phase, and thus the missing phase information gives an inaccurate result (that is, not equal to 
the focused case of the Fourier transform raised to the power 2) for what should have been the 
effective correction. 

Relative Phase Restoration 
[0096] In accordance with some embodiments, missing phase information may be 

reintroduced by absorbing it within the math correction stage, as shown block 905. This 
absorbing technique results in a phase-restored math correction of the form: 

where the following symbolic notation is used: 
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<D(X)= phase (X) (33) 
[0097] In the case where y is close enough to be effectively equal to 8, the phase 

correction will effectively be equal to the value necessary to restore the lost relative phase 
information. Note that this expression depends only on y, and thus phase correction may be 
obtained by systematically iterating y towards the unknown value of 8, which is associated with 
the misfocused image. Thus the iteration, computation, manual adjustment, and automatic 
optimization systems, methods, and strategies of non-phase applications of image misfocus 
correction may be applied in essentially the same fashion as the phase correcting applications of 
image misfocus correction by simply substituting F' with <D( F 2 ') ? in iterations or manual 
adjustments. 

[0098] Figure 10 provides an example of image information flow in accordance 

with some embodiments of the invention. This embodiment is similar to Figure 6 is many 
respects, but the technique shown in Figure 10 further includes phase restoration component 
1001 coupled with focus correction component 603. In operation, image array 602 is passed to 
phase restoration component 1001, which pre-operates on image array 602 . After the pre- 
operation calculation have been performed, fractional Fourier transform operation 603 is applied 
to the image array. 

Numerical Calculation of Relative Phase Restoration 
[0099] Next, the calculation of the phase-restored mathematical correction is 

considered. Leveraging two-group antislavery properties of the fractional Fourier transform 
operation, the additional computation can be made relatively small. 
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[0100] In the original eigenfunction/eigenvector series definitions for both the 

continuous and discrete forms of the fractional Fourier transform of power a, the nth 
eigenmnction/eigenvectors are multiplied by: 



^-inna/2 (34) 



Using this equation and replacing a with (2 - y) gives: 



for both the continuous 
following equation: 



e -inii(2-f)/2 _ e -inn ^-iwiH^ ( 35 ) 

and discrete forms of the fractional Fourier transform. Note that the 



e -innH) (36) 



can be rewritten as: 

g -M-y) _ jnict _ (g"'"^)* (37) 

where (X)* denotes the complex conjugate of X. 

[0101] Also, because the nth Hermite function h n (y) is odd iny for odd n, and 

even iny for even n, such that: 

h n (-y) = (-\) n h n (-y) (38) 

so that in the series definition the nth term behaves as: 

h n (x)h n (y)e^ /2 = h^hnW-iye^* ^ 

= h n (x)h n (-y)(e inK1 )* 

For both the continuous and discrete forms of the fractional Fourier transform, replacing h n (y) 
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with h„(-y) is equivalent to reversing, or taking the mirror image, of h n (y). In particular, for the 
discrete form of the fractional Fourier transform, this amounts to reversing the order of terms in 
the eigenvectors coming out of the similarity transformation, and because of the 
even-symmetry/odd-antisymmetry of the Hermite functions and the fractional Fourier transform 

discrete eigenvectors, this need only be done for the odd number eigenvectors. 

[0102] Further, since the Hermite functions and discrete Fourier transform 

eigenvectors are real-valued, the complex conjugate can be taken on the entire term, not just the 

exponential, as shown by: 

K (x)h n (-y)(e ^ / = [ h n (x)h n (-y) e** 1 ]* ( 40 ) 
Since complex conjugation commutes with addition, all these series terms can be calculated and 
summed completely before complex conjugation, and then one complex conjugation can be 
applied to the sum, resulting in the same outcome. 

[0103] The relative phase-restored mathematical correction can thus be calculated 

directly, for example, by the following exemplary algorithm or its mathematical or logistic 
equivalents: 

1 . For a given value of y, compute F r using the Fourier transform eigenvectors in an ordered 

similarity transformation matrix; 

2. For the odd-indexed eigenvectors, either reverse the order or the sign of its terms to get a 
modified similarity transformation; 

3. Compute the complete resulting matrix calculations as would be done to obtain a fractional 
Fourier transform, but using this modified similarity transformation; 

4. Calculate the complex conjugate of the result of operation (3) to get the phase restoration, 
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(0(F r ))*; and 

5. Calculate the array product of the operation (1) and operation (4) to form the phase-restored 

y * y 

focus correction (G>(F )) F . 
[0104] As an example of a mathematical or logistic equivalent to the just 

described series of operations, note the commonality of the calculations in operations (1) and (3), 
differing only in how the odd-indexed eigenvectors are handled in the calculation, and in one 
version, only by a sign change. An example of a mathematical or logistic equivalent to the above 
exemplary technique would be: 

1 . For a given value of y, partially compute F r using only the even-indexed Fourier transform 

eigenvectors; 

2. Next, partially compute the remainder ofF r using only the odd-indexed Fourier transform 
eigenvectors; 

y 

3 . Add the results of operation ( 1 ) and (2) to get F 

4. Subtract the result of operation (2) from the result of operation (1) to obtain a portion of the 
phase restoration; 

5. Calculate the complex conjugate of the result of operation (4) to obtain the phase 
restoration (<D(F r ))*; and 

6. Calculate the array product of operations (1) and (4) to form (0(F r )) F Y . 

[0105] In many situations, partially computing two parts of one similarity 

transformation, as described in the second exemplary algorithm, could be far more efficient than 
performing two full similarity transformation calculations, as described in the first exemplary 
algorithm. One skilled in the art will recognize many possible variations with differing 
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advantages, and «ha, these advantages may a.so vary with differing eontpntationa, architectures 
and processor languages. 

Embedding Phase Restoration within Image Misfocus Correction 
,0,06) Where relative phase-restoration is required or desired in mathematical 

focus correction using the fractional Fourier transform, phase restoration element 1001 may be 
used in combination with focus correetton element 603, as depicted in Figure 10. 

,0107! It is to be realized that in image misfocus correction applications which do 

no, account for phase restoration, pre-computed values off may be stored, fetched, and 
multiplied as needed or desired. Similarly, in tmage misfocus correction applications which 
provide for phase restoration, pre-computed values of *( F'»* F> may also be stored, fetched, 
and multiplied as needed or desired. For example, pre-computed value, of phase reconstructions 
m ay be stored corresponding to powers of the fractiona. Fourier transform, such mat the powers 
are related by roots of me number 2, or realized in correspondence to binaty representations of 
fractions, or both. In these compositions, care may need to be taken stnee the array 
multiplications may no. freely commute due to the nonlinear phase extraction steps. 

,01081 Each of the various techniques for computing the phase-restored focus 

correction may tnelude differing methods fo, implementing pre-computed phase-restorations. 
For example, in comparing the firs, and second exemplary algorithms, predominated values may 

be made and stored for any of: 
. First example algonthm operation (5) or its equivalent second example algonthm operation 

(6); 
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, n a , or its equivalent second example algorithm operation 
. First example algorithm operation (4) or its equivaie 

(5); and 

nUnHm wit h additional completing computations 
. Second example algorithm operations (1) and (2) with addioo 

provided as needed. 

,0,09] Again, it is noted tha, these phase restoration technicmes can apply to any 

and the gl oha, or ,oca,ized correcfion of nrisfoeus front elecfron tnicroseopy images laclcmg 
phase —ton. U caliz ed phase-restored m ,sfoc„s eorreetion „s,n g the t eehn iq ues disposed 

herein ntay he parficolarty usefia, in three-dimensional, electron ntieroscopy and tontography 
where a wide field is involved in a. leas, one dimension of imaging. 

, 011 01 While the invention has heen described in detad with reference to 

.se.osedemh.dimenu.variousmodificafionswithinthescopeofthetnventionwtnheapparent 

invention property is to be constrned with reference to the Cairns. 
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